Interactive Facial Image Retrieval

Interactive image retrieval is a powerful tool for image queries without input examples. For example, it is useful when searching a photo of a specified criminal only through the recalling of the witness. Unlike traditional text search engine working on input strings or keywords, a system which supports image query is usually required to learn opinions from users by relevance feedback and the retrieval is done in an interactive manner. Markus Koskela has conducted pioneering work on interactive image retrieval based on Self-Organizing Maps (SOM) [6]. The framework proposed by Koskela is able to learn relevance feedback from users in the ways of both intra-queries and inter-queries. The work has been incorporated in a system name PicSOM, which can return a set of desired images after a couple of interactive rounds. This way the semantic gap between human evaluation and low-level feature metrics can be reduced to some extent. The goal of my research is to further the work of Koskela and to make it closer to practical application in the following aspects: (1) focus on the case study of facial image retrieval; (2) improve average precision by combining multiple features, e.g. features from different facial parts in addition to that from whole face or features from multiple resolutions; (3) investigate the possibility of equipping the system with some high level features such as gender or race and the query performance when such high level features are present. The first task is to extract the face box and facial parts, which is identified as face detection problem and has been widely researched in recent years. From early template matching to recent techniques based on neural networks [12] and information theory [7], a detailed survey can be found in [14]. Lately statistical learning methods [10, 8] are reported to classify the face and nonface intensities successfully. After faces are extracted, the method of eigenface [5, 11] and its variants have already shown great success in the field of face recognition. When the input example is available, the matching precision by eigenface feature is nearly 90 percent. And advanced techniques [3, 9] can achieve even higher rate. For queries without input example, if one of the goal images is found, then the remaining query will be identical to those with input example. Thus the critical yet challenging point is to improve the average precision so that one of the target images can be shown as early as possible. Ruiz et. al. have proposed a SOM-based system, FACERET [2], for interactive facial image retrieval, where only eigen features of the whole face is used. On the other hand, it has been shown that a more accurate classifier can be obtained by combining multiple rough classifiers [4]. Since face recognition is a special case of classification problem by retrieving the images that belong to the same subject, we believe that the performance of interactive image retrieval system can be also improved by combining different features. Another obstacle of Content-Based Image Retrieval (CBIR) is the semantic gap. It is difficult for users to submit their query desire by the language of low-level features such as texture or eigen components. An approach of reducing this gap may be automatically extracting some high-level binary features and filtering the candidates to be displayed at each round. Numerous classification algorithms are available for our use, among which Kernel Discriminant Analysis (KDA) [1] and Support Vector Machines (SVMs) [13] are distinguished statistical learning methods and may be good options for our system. The first goal of my research is to construct a browsing system for interactive facial image retrieval. Afterwards, the above attempts of improvement on relevance feedback will be investigated and incorporated into the database. A further goal is to reduce the semantic gap between high level query requirement and low level facial features such that the system can be ready for some real applications.